UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition

Multimodal sentiment analysis (MSA) and emotion recognition in conversation (ERC) are key research topics for computers to understand human behaviors. From a psychological perspective, emotions are the expression of affect or feelings during a short period, while sentiments are formed and held for a longer period. However, most existing works study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. In this paper, we propose a multimodal sentiment knowledge-sharing framework (UniMSE) that unifies MSA and ERC tasks from features, labels, and models. We perform modality fusion at the syntactic and semantic levels and introduce contrastive learning between modalities and samples to better capture the difference and consistency between sentiments and emotions. Experiments on four public benchmark datasets, MOSI, MOSEI, MELD, and IEMOCAP, demonstrate the effectiveness of the proposed method and achieve consistent improvements compared with state-of-the-art methods.Comment: Accepted to EMNLP 2022 main conferenc

UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt

Recent research has shown that multi-task pre-training greatly improves the model's robustness and transfer ability, which is crucial for building a high-quality dialog system. However, most previous works on multi-task pre-training rely heavily on human-defined input format or prompt, which is not optimal in quality and quantity. In this work, we propose to use Task-based Automatic Prompt generation (TAP) to automatically generate high-quality prompts. Using the high-quality prompts generated, we scale the corpus of the pre-trained conversation model to 122 datasets from 15 dialog-related tasks, resulting in Universal Pre-trained Conversation Model (UniPCM), a powerful foundation model for various conversational tasks and different dialog systems. Extensive experiments have shown that UniPCM is robust to input prompts and capable of various dialog-related tasks. Moreover, UniPCM has strong transfer ability and excels at low resource scenarios, achieving SOTA results on 9 different datasets ranging from task-oriented dialog to open-domain conversation. Furthermore, we are amazed to find that TAP can generate prompts on par with those collected with crowdsourcing. The code is released with the paper

UniSA: Unified Generative Framework for Sentiment Analysis

Sentiment analysis is a crucial task that aims to understand people's emotional states and predict emotional categories based on multimodal information. It consists of several subtasks, such as emotion recognition in conversation (ERC), aspect-based sentiment analysis (ABSA), and multimodal sentiment analysis (MSA). However, unifying all subtasks in sentiment analysis presents numerous challenges, including modality alignment, unified input/output forms, and dataset bias. To address these challenges, we propose a Task-Specific Prompt method to jointly model subtasks and introduce a multimodal generative framework called UniSA. Additionally, we organize the benchmark datasets of main subtasks into a new Sentiment Analysis Evaluation benchmark, SAEval. We design novel pre-training tasks and training methods to enable the model to learn generic sentiment knowledge among subtasks to improve the model's multimodal sentiment perception ability. Our experimental results show that UniSA performs comparably to the state-of-the-art on all subtasks and generalizes well to various subtasks in sentiment analysis.Comment: Accepted to ACM MM 202

Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models

Task-oriented dialogue (TOD) systems facilitate users in executing various activities via multi-turn dialogues, but Large Language Models (LLMs) often struggle to comprehend these intricate contexts. In this study, we propose a novel "Self-Explanation" prompting strategy to enhance the comprehension abilities of LLMs in multi-turn dialogues. This task-agnostic approach requires the model to analyze each dialogue utterance before task execution, thereby improving performance across various dialogue-centric tasks. Experimental results from six benchmark datasets confirm that our method consistently outperforms other zero-shot prompts and matches or exceeds the efficacy of few-shot prompts, demonstrating its potential as a powerful tool in enhancing LLMs' comprehension in complex dialogue tasks

Detection of Flare-induced Plasma Flows in the Corona of EV Lac with X-ray Spectroscopy

Stellar flares are characterized by sudden enhancement of electromagnetic radiation from the atmospheres of stars. Compared to their solar counterparts, our knowledge on the coronal plasma dynamics of stellar flares and their connection to coronal mass ejections (CMEs) remains very limited. With time-resolved high-resolution spectroscopic observations from the \textit{Chandra} X-ray observatory, we detected noticeable coronal plasma flows during several stellar flares on a nearby dMe star EV Lac. In the observed spectra of O~{\sc{viii}} (3 MK), Fe~{\sc{xvii}} (6 MK), Mg~{\sc{xii}} (10 MK), and Si~{\sc{xiv}} (16 MK) lines, these flare-induced upflows/downflows appear as significant Doppler shifts of several tens to \speed{130}, and the upflow velocity generally increases with temperature. Variable line ratios of the Si~{\sc{xiii}} triplet reveal that these plasma flows in most flares are accompanied by an increase of the coronal plasma density and temperature. We interpret these results as X-ray evidences for chromospheric evaporation on EV Lac. In two successive flares, the plasma flow pattern and a sharp increase of the measured coronal density are highly suggestive of explosive evaporation. The transition from redshifts to blueshifts in such an explosive evaporation occurs at a temperature of at least 10 MK, much higher than that observed in solar flares ($\sim$1 MK). However, in one flare the cool and warm upflows appear to be accompanied by a decreasing plasma density, which might be explained by a stellar filament/prominence eruption coupled to this flare. These results provide important clues to understand the coronal plasma dynamics during flares on M dwarfs.Comment: accepted by Ap

CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation

Practical dialog systems need to deal with various knowledge sources, noisy user expressions, and the shortage of annotated data. To better solve the above problems, we propose CGoDial, new challenging and comprehensive Chinese benchmark for multi-domain Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources: 1) a slot-based dialog (SBD) dataset with table-formed knowledge, 2) a flow-based dialog (FBD) dataset with tree-formed knowledge, and a retrieval-based dialog (RBD) dataset with candidate-formed knowledge. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing. The proposed experimental settings include the combinations of training with either the entire training set or a few-shot training set, and testing with either the standard test set or a hard test subset, which can assess model capabilities in terms of general prediction, fast adaptability and reliable robustness.Comment: EMNLP 202

Hypothermia in Stroke Therapy: Systemic versus Local Application

Presently, there are no effective, widely applicable therapies for ischemic stroke. There is strong clinical evidence for the neuroprotective benefits of hypothermia, and surface-cooling methods have been utilized for decades in the treatment of cerebral ischemia during cardiac arrest, but complications with hypothermia induction have hindered its clinical acceptance in ischemic stroke therapy. Recently, the microcatheter-based local endovascular infusion (LEVI) of cold saline directly to the infarct site has been proposed as a solution to the drawbacks of surface cooling. The safety and efficacy of LEVI in rat models have been established, and implementation in larger animals has been similarly encouraging. A recent pilot study even established the safety of LEVI in humans. This review seeks to outline the major research on LEVI, discusses the mechanisms that mediate its superior neuroprotection over surface and systemic cooling, and identifies areas that warrant further investigation. While LEVI features improvements on surface cooling, its core mechanisms of neuroprotection are still largely shared with therapeutic hypothermia in general. As such, the mechanisms of hypothermia-based neuroprotection are discussed as well

Improving Factual Consistency of Text Summarization by Adversarially Decoupling Comprehension and Embellishment Abilities of LLMs

Despite the recent progress in text summarization made by large language models (LLMs), they often generate summaries that are factually inconsistent with original articles, known as "hallucinations" in text generation. Unlike previous small models (e.g., BART, T5), current LLMs make fewer silly mistakes but more sophisticated ones, such as imposing cause and effect, adding false details, overgeneralizing, etc. These hallucinations are challenging to detect through traditional methods, which poses great challenges for improving the factual consistency of text summarization. In this paper, we propose an adversarially DEcoupling method to disentangle the Comprehension and EmbellishmeNT abilities of LLMs (DECENT). Furthermore, we adopt a probing-based efficient training to cover the shortage of sensitivity for true and false in the training process of LLMs. In this way, LLMs are less confused about embellishing and understanding; thus, they can execute the instructions more accurately and have enhanced abilities to distinguish hallucinations. Experimental results show that DECENT significantly improves the reliability of text summarization based on LLMs

Molecular doping enabled scalable blading of efficient hole-transport-layer-free perovskite solar cells

The efficiencies of perovskite solar cells (PSCs) are now reaching such consistently high levels that scalable manufacturing at low cost is becoming critical. However, this remains challenging due to the expensive hole-transporting materials usually employed, and difficulties associated with the scalable deposition of other functional layers. By simplifying the device architecture, hole-transport-layer-free PSCs with improved photovoltaic performance are fabricated via a scalable doctor-blading process. Molecular doping of halide perovskite films improved the conductivity of the films and their electronic contact with the conductive substrate, resulting in a reduced series resistance. It facilitates the extraction of photoexcited holes from perovskite directly to the conductive substrate. The bladed hole-transport-layerfree PSCs showed a stabilized power conversion efficiency above 20.0%. This work represents a significant step towards the scalable, cost-effective manufacturing of PSCs with both high performance and simple fabrication processes